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Abstract 

Reports of flipped classrooms across areas in the field of ESL are rare, and those that 
address the complexities of ESL teacher education, particularly the methods course in 
which procedural knowledge is privileged over declarative knowledge, are even rarer. This 
paper uses a design-based research (DBR) approach to explore the flip of an ESL teacher 
education course for mainstream teachers. This study, focusing on the five members of the 
design team, seeks to fill several gaps in the literature: the lack of design specifics; data on 
the use of flipped instruction in teacher education; the need for examination of flipped 
teacher education contexts. The paper, formatted in the style of DBR, provides suggestions 
for design steps, data sources, and principles for the design of teacher education courses. 

Introduction 

Traditionally, much of class time at all levels of education is spent with the teacher 
explaining concepts through lecture. Students listen and take notes, and then they take 
those notes home to refer to while working through homework assignments. This system 
can create passive learners who may struggle through tasks unassisted (Shimamato, 2012), 
an issue that has been noted in some teacher preparation programs. All students, including 
teacher education students, need to perceive instruction as useful, stimulating, and 
providing the best use of classroom time. One way to support this perception for teacher 
education students may be to focus on resource-rich, student-centered learning, and one 
interesting and relatively new approach that supports this type of learning is flipped 
instruction. 
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In the flipped classroom, instructors (and expert others) provide access to direct 
instruction resources for students outside of school (Strayer, 2007) so that class time can 
be spent more on inquiry-based and experiential learning (Berrett, 2012). After accessing 
direct instruction outside of class, teacher education students can come to class ready to 
explore, elaborate on the content, and interact with each other in ways that meet their 
learning needs, instead of just spending seat time. Overall, the flipped instruction model 
provides a collaborative learning environment in which students can work through course 
materials with their peers and with the assistance of the instructor without sacrificing 
direct instruction (Bergman & Sams, 2012; Foertsch, Moses, Strikeweda, & Litzkow, 2002; 
Kim, Byun, & Lee, 2012; Strayer, 2007). Because of its focus on procedural knowledge and 
collaboration, this approach seems to show promise as an instructional option for teacher 
education programs. 

Because there are few guidelines for or deep, evidence-based descriptions of successful 
flipped classrooms, there are gaps in the literature that need to be filled. This paper begins 
to fill the gap by first briefly reviewing the literature on teacher learning and flipped 
instruction and then by discussing the design-based framework used for this study. This 
paper then presents results and discussion of the design and implementation of the focus 
course and concludes with implications for research and teaching. 

Literature 

Teacher Learning 

The effective use of any instructional strategy depends in great part on understanding how 
the target population learns. Although Khan (2010) and others appear to support the 
common public understanding that teaching means in part being able to transmit 
information to students, the literature on teacher education shows that this informational 
teaching "cannot suffice as ways of conceptualizing practice in the world of teaching about 
teaching" (Laughran, 2013, p.15). Instead, effective teachers rely on variably applied 
literacies rather than reproduction of disciplinary facts (Clarke, 2013). Clearly, disciplinary 
content is a necessary foundation for teaching (Ball, Thames, & Phelps, 2008), but learning 
to teach also means supporting the use of such content in teacher-like ways (Feiman- 
Nemser & Remillard, 1995). In other words, learning to teach means being able to draw on 
content, task and dispositional knowledge to deal with authentic classroom dilemmas 
spontaneously, or to use "thinking-in-action” (Schon, 1987). This way of teaching is quite 
different from passing on a body of knowledge that students would memorize. Similarly, 
Darling-Hammond (2008) emphasizes that teachers do not only need in-depth knowledge 
of subject matter, but they also need to be able to understand it flexibly, relating it to 
everyday life so that they can make it accessible to students with different experiences and 
interests in different classroom contexts. 

These new strategies are not easy to learn or do; they take teachers’ time, practice, and 
repeated exposure to the ideas. It also appears that understanding how to implement 
learner-centered pedagogical strategies such as problem-solving, flexibility, just-in-time 
decision making, and a host of other dispositions needed every day in classrooms cannot be 
taught to teachers through traditional methods (Prince & Felder, 2006), which suggests 
that lecture alone cannot easily help teacher education students to learn these skills 
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(McKeon, 1994; Stoddart, Pinal, Latzke, & Canaday, 2002). Furthermore, providing teacher 
education students with engaging instruction that involves active learning is important, 
since novice teachers tend to teach based on their experiences as students (Grippin, 1989). 
Thus, modeled active teaching practices can help pre-service teachers learn and practice 
them more easily, and these models can also provide hands-on experience with good 
teaching practices (Loughran & Berry, 2005). Whether learning can occur more effectively 
through flipped teacher education instruction is a question to be explored. 

Flipped Instruction 

There are yet no set guidelines for exactly what flipped instruction should look like. 
Although not a new concept, the movement has recently increased in popularity 
throughout education, with entire public schools and even districts applying their own 
versions of flipped instruction (e.g., Flanigan, 2013; see Brinks Lockwood, 2014, for flipping 
ESL classrooms). Much of the early work in flipped instruction has taken place within 
disciplines that in the literature are considered to be based mostly on declarative and 
structural knowledge (Donald, 1983). However, little research has been conducted in 
complex contexts like teacher education, which are based mainly on relatively open-ended 
procedural knowledge (see Heintin, 2013, for an example of teacher professional 
development). 

The purpose of flipped instruction is generally to utilize technology to increase 
opportunities for teacher-student and student-student interaction by changing the dynamic 
in which information is presented (Baker, 2000). In its most basic form, flipped instruction 
consists of pre-recorded direct-instruction lecture content which is made available online 
for students to access at home. Students watch these videos as homework before class and 
learn the content lecture on their own time, which allows for class time to be spent on 
other activities such as group projects, labs, and discussions (Bergman & Sams, 2012; 
Baker, 2000; Kim, Byun, & Lee, 2012; Lage, Platt, & Treglia, 2000). According to the extant 
literature, a flipped classroom is based on what the research tells us works: 

• meaningful activities instead of busy work 

• teacher as tutor rather than director 

• increased interaction around content 

• a focus on learning, not just behaving in a "school" way (e.g., sitting in a chair 
listening to the teacher) 

• immediate feedback for students on process and progress 

• making technology integral to learning 

• just-in-time instruction. 

In addition, in flipped classrooms, students have more time to spend with the content 
presentation (e.g., a PowerPoint presentation, a screencast, a book, or a podcast), and they 
can rewind and/or reread if they want to access the content again or speed through what 
they already understand. This type of student control can have a positive effect on learning 
as it frees up class time for students to explore the ideas in hands-on ways that align with 
their needs and abilities. 
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Elements of Flipped Instruction 

Although mostly anecdotal, reports on flipped classrooms lead to a list of elements that 
generally characterize a flipped approach. These elements are described below. 

Digital lectures. The use of instructional videos is a key element in the makeup of flipped 
instruction. These videos can be created by the teacher, students, or someone hired to 
make them, or they may be found online and used with little or no modification as 
appropriate. In a flipped approach, the videos are typically made available to students 
online for easy access outside of class. 

Course management system (CMS). Some instructors use a CMS to assist in the 
management of educational courses for students. CMSs can be used to store course syllabi, 
present course announcements, manage student email, provide online discussion, support 
online submission of assignments, and host teacher- and student-posted videos. CMSs have 
proven to be an important aspect of some, but not all, flipped classrooms (Strayer, 2007). 

Textbooks and outside texts. In some classrooms, the textbook is one of several 
resources provided to the students to meet the course objectives. The textbook typically 
complements the videos and is intended to be used alongside the videos and other 
resources for out-of-class preparation (see, for example, Bergmann & Sams, 2012). In other 
classrooms, the instructional videos replace textbooks completely (see, for example, 
Foertsch, Moses, Strikeweda, & Litzkow, 2002; Lage, Platt, & Treglia, 2000; Strayer, 2007). 

Group discussions and group work. Teachers who require students to bring lecture- 
related questions to class often began class with a discussion of those questions (Bergmann 
& Sams, 2012), designed to clear up any confusion or misconceptions that may have come 
from the using the videos and/or resources. Another common task during class time in 
many flipped classrooms is group work and/or lab activities. These tasks provide 
opportunities for collaborative learning and problem-solving (Shimamoto, 2012; Strayer, 
2007), as students share perspectives and gain insights that they might otherwise be 
unable to obtain on their own. 

Mini-lessons. Mini-lessons can be based on student-generated questions and group 
discussion. If enough students have the same question after interacting with class 
resources, the teacher may briefly reteach that particular aspect of the lecture (Bergmann 
& Sams, 2012; Lage, Platt, & Treglia, 2000; Strayer, 2007). 

Differentiated pacing and timing. In some flipped classrooms, students work at their own 
pace, so not all students are working on the same task at any given time. For instance, 
teachers may group students based on the task that they are currently working on, which 
allows students to have peer support while giving them the freedom to work at their own 
pace. This technique allows teachers to more effectively address student needs, as students 
in a particular group may be struggling with similar concepts from a given lesson 
(Bergmann & Sams, 2012). 

Studies of Flipped Instruction 

At present, the few studies conducted on flipped instruction consist mainly of self-report 
data; however, these findings provide a foundation for future exploration. In general, 
studies have shown that students gave favorable marks to the implementation of the 
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flipped model in their classrooms (Bergman & Sams, 2012; Foertsch, et al, 2002; Strayer, 
2007; Kim, Byun, & Lee, 2012; Lage, Platt, & Treglia, 2000; Mazur, 1991). Students claimed 
to have appreciated the extra attention they received from their instructor and peers, and 
they also valued the ability to review lectures and view them at a time and place of their 
choosing (Bergman & Sams, 2012; Foertsch, et al, 2002; Kim, Byun, & Lee, 2012; Lage, Platt, 
& Treglia, 2000; Strayer, 2007). Those students who struggled with the flipped model 
found that they needed to be much more independent in their learning (Foertsch, et al, 
2002; Lage, Platt, and Treglia, 2000; Strayer, 2007) and noted that the dramatic change 
between teaching models proved to be a difficult adjustment (Strayer, 2007). Nevertheless, 
more systematic study is required in a variety of contexts to understand the relationship of 
these perceptions to aspects of the learning environment and to theoretical models of 
instruction. 

In the context of flipped teacher education, anecdotal and informally gathered data 
presented in the literature (e.g., in blogs, online teacher discussion, video, teaching practice 
web sites) suggests that, when done well, flipping instruction can lead to: 

• better and more effective learning; 

• improved grades due to more student-centered learning opportunities; 

• increased student engagement and concurrent decreased classroom disruptions; 

• increase in test scores from greater focus on understanding; 

• alignment of student needs and school resources. 

Certainly other instructional techniques are built on many of the same principles as 
flipping and have achieved success in the hands of competent instructors. Different in the 
flipped strategy is that teacher education students can have control over the content 
delivery that they do not normally have in traditional classrooms; the flipped classroom, 
instead, focuses on every student mastering the course content. In teacher education, it is 
absolutely crucial for every student master the skills and dispositions of teaching. However, 
like any type of instruction, flipped instruction can be done well, or it can fail miserably 
(Strayer, 2007). We do not yet know how the elements of flipped instruction can and 
should be implemented in different classrooms, as we noted above, because teacher 
education can be viewed as more complex and less concrete than some other disciplines. 

Method 

The exploratory study described below provides data on the design and implementation of 
a flipped teacher education classroom. It seeks to fill several gaps in the literature, such as 
the lack of design specifics, the need for data on the use of flipped instruction in teacher 
education, and the necessity for examination on teacher education contexts. The research 
team attempted to follow the extant flipped literature as closely as possible to test this 
application in teacher education classrooms. To examine this implementation, the study 
applied a design-based methodology to examine the process and the underlying reasoning 
for it and to explain changes and adaptations to the course that may support other 
designers in their process. The following questions guided the study: 
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• How do teacher education students react to a change in the format of their 
instruction? How do they perceive and use the additional resources, including peers? 

• How do students and research team members perceive the implementation of flipped 
instruction in teacher education? What principles are important to them? 

Design-based Research 

Design-based research, commonly known as either design research, design experiment, or 
development research (van den Akker, 1999), is a systematic research methodology that 
provides a holistic view of the study of teaching and learning since it focuses on both 
learning theories and the development of instructional materials and strategies that are 
rooted in messy education settings (Brown, 1992; Collins, 1992; Design-Based Research 
Collective, 2003). Design-based research not only deals with practical issues in education, 
but also seeks to increase the impact of research through iterative testing and the 
refinement process of the effectiveness and sustainability of design principles and relevant 
theories (Anderson & Shattuck, 2012; Barab & Squire, 2004; Kelly, 2004; Parker, 2011). 

Although there are variations among design-based approaches, Anderson and Shattuck’s 
(2012) analysis of design-based research proposes that good design-based research 
follows the following eight criteria and: 

1. is situated in a real educational context; 

2. is focused on the design and testing of a significant intervention; 

3. uses mixed methods; 

4. involves multiple iterations; 

5. involves a collaborative partnership between researchers and practitioners; 

6. evolves design principles; 

7. differentiates from action research; 

8. has a practical impact on practice (p. 17-18). 

This study follows these criteria as closely as possible through the process described by 
Reeves (2006), which includes four interconnected phases: 1) analysis and exploration of a 
problem and creation of preliminary research questions (Herrington, Reeves, & Oliver, 
2010); 2) development of solutions using existing design principles and technological 
innovations; 3) implementation and evaluation in iterative cycles, including "continuous 
cycles of design, enactment, analysis, and redesign" (Design-Based Research Collective, 
2003, p. 5), and; 4) reflection to produce design principles that can be shared for further 
refinement by other practitioners and researchers. 

Context 

This study took place in the College of Education of a major university in the rural 
northwestern United States, and it was approved (exempted) by the institution’s review 
board. The two sections of the teacher education course being described, Introduction to 
ESL (English as a Second Language], each met once per week for 2.5 hours during the fall 
2012 and spring 2013 semesters. The focal context for the course was a class wiki, on 
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which detailed information about the course, assignments, tasks, resources, and dates was 
posted, as were discussion questions and student products such as lesson plans. 

Participants 

The research/design team in this study included two researchers (one of whom also served 
as the instructor of record for the course and will thus be called "researcher/instructor” in 
the rest of this paper). The first researcher was an expert in language learning and 
technology and also had some design experience; this researcher participated mainly in the 
discussion of the overall goals of the course being redesigned and also worked on the 
design. The researcher/instructor was also well-versed in the technology, teaching 
methodology, and task engagement literatures and supervised the design process. Three 
education doctoral students (one of whom was the teaching assistant for the course) and 
one post-doc completed the six-member research/practice team. The primary 
responsibilities for two of the doctoral students and the post-doc were to participate in 
course material development and perform classroom observations. The doctoral students 
and the post-doc all had been language teachers previously, had experience with a variety 
of technologies, and all were studying in the Language, Literacy, and Technology program. 
In general, the interdisciplinary team held positive views of flipped instruction as a strategy 
to create a resource rich, student-centered learning environment, and the team was very 
motivated to implement it with teach pre-service teachers. 

In addition, 106 teacher education students - 100 female and 6 male students (referred to 
as "students” or "participants” for the rest of this paper) - who were taking the Introduction 
to ESL course in their last semester of their teacher education degree program served as 
participants over two semesters. Although some students in the course had immigrant 
parents (e.g., Hispanic, Vietnamese, or Chinese), all of them spoke English as their first 
language. 

Data Sources and Analysis 

Data sources included both numerical and self-report measures to meet the requirements 
of design research and provide enough information for the team to make reasonable 
decisions. During both semesters of the course under investigation, the research/design 
team collected existing course documents (e.g., both online and hard copies of syllabi, wiki 
pages, and assignments), logs from the video site, student input including midterm course 
evaluations and intermittent course review questions, team meeting and field notes, 
classroom observations made by members of the design team, and student comments and 
questions, along with notes from the literature and available resources for flipped 
instruction. The research team initially met several times per week and emailed in 
between; notes were taken during these interactions. 

In addition, five of the team members attended all of the course sessions, which included 
two members as instructors and three as observers and participants. Students in the 
course were not told the specific purpose of the observations, but it was suggested that 
because all three observers were international students and participating in some of the 
course activities, they were learning to be better teachers. In order to gauge student 
perceptions of and reactions to the instruction and resultant changes in their 
responsibilities, the researcher/instructor created a midterm course evaluation, added a 
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review "quiz” question about their use of class resources to a content review "quiz,” and 
included questions about perceptions of the tasks in the Moodle and as part of the 
reflections for the fieldwork. Student reactions were also gauged by class observations and 
impromptu conversations with individual students. In addition, one of the team members/ 
observers was also the facilitator for the Chinese Moodle portion of the course, which is 
described below. Initially the observations were logged using a format in which the 
observers noted teacher/student talk time, classroom interaction patterns, and any student 
comments related to the instructional style of the class. After the first three course 
sessions, the logs were not used because the data were consistent across sessions. 
Therefore, after the third class, the observers recorded their own comments and students’ 
comments and questions about tasks and classroom events that could be changed to 
support greater interaction. These observations were then discussed with the research 
team. 

For the purposes of data analysis, every class session was seen as an iteration of the 
research design because of the standardized structure of assignments, instruction, and 
interactions. In addition, the second semester of the flipped version of the course was also 
seen as an iteration of the overall idea. The researchers conducted analysis on the 
qualitative and anecdotal data (e.g., observations, documents, evaluations, among others.), 
following coding guidelines adapted from Boeije (2010); coding included reading through 
all of the data to determine categories of changes, and then grouping design decisions 
according to category. Data from all sources that applied to that category (codes) were 
listed under that category to provide support for the changes made. The occurrences of 
specific codes were counted to add numerical support to the account. Numerical data, such 
as the number of times students watched the course videos, were recorded and totaled. 
Plausibility was enhanced by member-checking with the team and rechecking notes and 
other data sources. The central patterns linked to the important aspects of flipped learning 
(as noted in the literature review) and other salient and frequent aspects noted in the data 
were used to construct the narrative below. 

Narrative 

The results and discussion are presented below in the four categories described by Reeves 
(2006). These four steps represent the almost weekly iterations of design changes and 
incorporate the major design decisions, changes, and challenges encountered throughout 
this process. 

Problem 

The Introduction to ESL course is a 10-week required course that has been taught every 
semester for eight years to pre-service mainstream teachers in the teacher education 
program. The purpose of the course is to help prepare teachers to serve the needs of 
English language learners (ELLs) in mainstream classrooms, providing ELLs with ways to 
access content and achieve goals in both language and content. Although the course had 
been fairly effective in meeting these goals based on student evaluations and random 
follow-up from students as they became teachers, the faculty in the ELL program were 
often concerned about the lack of interaction between the pre-service teachers and actual 
ELLs and their families. In addition, because the great majority of the teacher education 
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students in the program had white, middle-class, Christian backgrounds, faculty were also 
apprehensive about the lack of exposure and experience of the teacher education students 
with diversity in general. A variety of informal changes to the course each semester were 
implemented to address these issues, but both faculty and students felt that more could be 
done. These shared concerns led to three goals for the redesign: 

• Involve teacher education students more in situations with ELLs 

• Support greater understanding of the components of learning that affect ELLs 

• Be more effective in engaging teacher education students in class 

Obstacles to instituting such change included lack of sufficient time in class to address 
other direct learning goals, problems with scheduling visits to K-12 classrooms in the small 
local school district or visitors to the class, difficulties in having students collaborate 
outside of class because of their tight schedules, and other administrative issues. As a result 
of these challenges, we decided to implement a new model for class structure and content. 

A review of the limited literature on flipped instruction by the researchers initiated both 
interest and skepticism, as well as an application for a small university-based grant to 
implement this approach in the ESL methods class. The funding allowed the project team to 
be compiled and the design process to begin. The design team first came to a very general 
understanding of flipped instruction by reading the extant literature. In trying to explicate 
our approach, we found that the difference between a regular blended classroom and a 
flipped classroom was a difference between format and content. For example, while flipped 
learning uses the format of a blended approach by incorporating aspects of e-learning, 
flipped learning is more specifically structured in that it requires that most direct 
instruction be completed by students outside of class. For what we perceived to be ease of 
implementation, we decided to focus on "traditional" flipping (i.e., moving the whole class 
along at the same pace) rather than "mastery" (i.e., allowing individual students to move 
through the course materials at their own pace) (Bergmann & Sams, 2012). Based on the 
research questions noted previously, the design team worked to develop design solutions 
that would address all of the goals and questions for the class; this is the goal of design- 
based research. 

Development of Solutions 

Through an integration of the data, team members’ knowledge and experiences, references 
to the literature, exploration of the existing course, and available resources, the team 
sought to address the research questions. 

Involving students with ELLs. First, the two researchers discussed ways in which the 
course could be structured to allow students to have field experiences around ELLs. 
Previous student complaints about the length of each class, scheduled for 4.5 seat hours in 
one sitting per week, supported the researchers’ desire to break up the time and provide 
students with effective situated experiences rather than lecture and discussion in class 
with application for homework. They agreed that if direct instruction could be moved 
outside of the class time, one hour of seat time per week could be dedicated to fieldwork. 
The researchers discussed the types of fieldwork that were both available and useful, in 
addition to the limitations given the rural location of the university. 
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An additional thought was that students needed choices, not only to help them find 
something engaging for them but also in order to provide tasks that would fit in with their 
other coursework, their time in other locations during university breaks, and so on. The 
result was a set of nine tasks created and edited by the researchers, three in each category 
of community, parents, and students. Tasks included activities such as observing a local K- 
12 classroom that included ELLs, attending a culture-based activity at the university, and 
visiting, observing, and/or visiting churches, stores, and restaurants where a majority of 
minority-language learners might be found. Students were asked to choose one task to 
accomplish in each category and turn in a description (i.e., text, video, slides, or another 
format) and reflection on the due date. By moving the direct instruction out of class for the 
flip, time was made to incorporate this fieldwork as part of "class" time. 

A second part of the course was a Chinese for Teachers online Moodle that was integrated 
into the Introduction to ESL course to provide teacher education students with insight into 
how their future ELLs might feel facing a new language and content in school situations. 
The course consisted of a series of online modules that presented direct instruction in 
Chinese language and culture through English and Chinese and included readings, writing 
exercises, and some videos. Much of the content was in English, but participants still 
struggled with learning an unfamiliar, and what they perceived as difficult, language. This 
struggle was clearly expressed by many students in the Moodle discussion in the previous 
iteration of the course. For example, one student that wrote what others echoed: 

It is very easy to look at the word and know what it is supposed to sound like but it 
very rarely comes out the way that I want it to. I can never quite determine if I am 
adding emphasis to the wrong parts of the words or not. Listening back I never know if 
what I am saying makes sense or hold some unknown emotion. The tones are very 
difficult for this native English speaker to say the least. 

Students were expected to spend approximately an hour per week outside of class 
completing the reading, writing, and culture activities, and participating in online 
discussions that linked their Chinese learning to concepts in the rest of the ESL course; in 
fact, in the end of the semester evaluation, students reported spending from 30 minutes to 
two hours on their work, depending on the unit. This assignment started during the third 
week of class and lasted for seven weeks. Time "off” from the total class seat time was 
provided for this activity so that it could be done outside of class; flipping the whole class 
allowed us to integrate this new aspect. 

A final proposed solution to better increase students’ exposure to ELLs was to encourage 
international and ELL students from across the university, including doctoral students 
from the research team, to attend the on-site class and share their insights during 
interactive mini-lessons on language teaching strategies that the students in the class were 
required to present. Twice during the semester, several K-8 students also attended and 
participated in the mini-lessons. In the end-of-course evaluations and informal feedback 
received throughout both semesters, participants commented that having ELLs in the class 
was very helpful and interesting, particularly because the format of the course allowed the 
participants to interact with the ELLs rather than just hypothesize about them. In other 
words, flipping the direct instruction outside of the class created more opportunities for 
interaction and authentic application during class time. 
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Overall, the design team believed that these changes, particularly the introduction of field 
work, would help to meet the course goals and provide more exposure to ELLs in ESL 
contexts. These changes were made possible by flipping the course instruction. 

Greater understanding of concepts. Following the flipped literature, the team decided to 
produce a series of videos that would simplify and exemplify the course text’s main 
ideas. Corel VideoStudio Pro X4 was chosen as the central design technology based on its 
ease of use, flexibility, and ability to integrate easily with Screencast.com, a Web-based 
video storage platform. The team expected that the videos would help students determine 
the main ideas of the text and serve as a central resource for students who have visual 
and/or audio learning preferences. A list of possible topics was drawn up by the 
researchers, and a format for the videos was created by the researcher/instructor based on 
the technology chosen. Both the list and format changed many times before being finalized, 
and changes depended both on the technologies available and accessible for students and 
the desire to make the videos 5-8 minutes long, following recommendations in the flipped 
literature (actual videos ranged from 5-14 minutes, and each lesson contained between 
two and four videos). Four of the project team members created videos, which this also 
affected how the videos were constructed. For instance, one of the team members included 
pop culture references and created an informal voiceover, while other members’ videos 
sounded more formal and academic. Because the design team aimed to both assess 
whether students watched the videos and what they understood from them, one to three 
short comprehension quiz questions were added to each video; the videos were then 
posted to the free version of Screencast (screencast.com) so that the quizzes could be taken 
online. Screencast would record students’ answers and scores. Figurel shows a screenshot 
of some of the quizzes in the Screencast folder. 
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AS 1 Assessment Background 


AS 2 Types of Assessments 


AS 3 Helping ELLs Take Tests 


BG 1 ELLs and the Law 



BG 2 ELL Demographics 


Figure 1. Some of the quizzes in Screencast. 


The video logs data show that students used the videos in the ways expected and for the 
reasons outlined. In the first semester, 16 of 55 student participants used only the videos to 
understand course content, stating in the question added to the reading quiz (see data 
sources) that they were "visual" learners. Six of the participants chose to use the textbook 
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alone, noting that they liked to highlight text and felt that the text was more thorough than 
the videos. The rest of the participants used both the videos and the text, but most added 
that they watched the videos first and then skimmed the text to answer any questions or 
reinforce the video concepts. 

Engagement. For the out-of-class course tasks noted above, we tried to create conditions 
common in the engagement literature (e.g., Dornyei, 2002; Meltzer & Hamman, 2005). 
These tasks were connected to student needs and interests, providing information in ways 
that students would find authentic while ensuring that students had the skills to meet the 
challenges of the tasks. For the in-class portion of the course, the flipped literature 
indicates that a discussion, followed by student-controlled interaction around tasks, would 
help to engage students by allowing them to work at their own pace, receive feedback 
when needed, and consult with peers and the teacher for just-in-time support (Berrett, 
2012; Fulton, 2010). Accordingly, we initially asked students to post one question per week 
about the class content to the course wiki; these questions would then be discussed in class 
using different groupings. In addition, during each class period two teams of students 
demonstrated a teaching strategy from the required teaching strategies book to the class so 
that their peers could understand the strategy in a semi-authentic context. The other 
central activity in the in-class portion included working alone or in groups on the lesson 
plans that were the main deliverable for the course. One student understood this format as 
"teaching ourselves,” although the instructor provided individualized feedback and 
suggestions during the lesson-building period of each class. 

Student reactions. The data provided a variety of perceptions of and reactions to the 
flipped instructional format, but several patterns were clearly evident. These reactions are 
integrated into the next section on implementation and evaluation. 

Implementation and Evaluation 

Problems with the flipped design fell into the categories of technical, process, and content 
challenges, and were sometimes related to each other. Some of the challenges arose 
because there were no choices in some of the technologies or in how they were supported 
(or not) by the university. Other challenges were a consequence of compromises among 
team members with differing opinions and outlooks. Because there was no one with 
experience in flipped instruction technologies and strategies at the university at the time of 
the study, the research team drew closely from the literature on flipped teaching. In spite of 
the successes, there were non-catastrophic difficulties that had to be worked out, which are 
described below. 

Technical difficulties. In a course that relies heavily on technology and learning outside of 
class, any glitches can sideline a task and disengage students. The design of the technology 
pieces was specifically meant to provide all of the resources in the course wiki, where 
students might easily find them. Nevertheless, students commented during class that they 
needed more links in different places so wherever they found themselves on the wiki they 
could navigate to where they wanted to be. The lack of links also included the Moodle, 
which initially was accessible only from the main wiki page of the course. After considering 
student comments, links to Moodle were added in every unit’s schedule on the wiki so 
students could reach all materials from the schedule page where the assignments were 
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posted (as in Figure 2 above). Other technical issues were based on individual students’ 
technology access. Some of them did not have microphones to record their pronunciation 
of the Chinese characters, and others could not post to the Moodle. Adding information to 
the wiki about preferred browsers (i.e., Chrome and Firefox) and making computer 
peripherals such as headphones available in class and for checkout solved most of these 
types of limitations. 

One very problematic technical issue was that the volume on some of the videos we created 
was too low when played on certain devices; in addition, the quizzes did not always work, 
and the Screencast site was found to be very unreliable. For the second half of the first 
iteration of the course and in subsequent iterations, students were excused from doing the 
quizzes and some students accessed the videos from the course YouTube site from which 
the quizzes would not run. No solution was found for the differences in sound quality, and 
the design team decided to look into it more closely after the 2nd semester iteration of the 
course completed. 

In addition, the ability of the wiki platform to collect student data was limited. It was clear 
when students posted a document (students posted from 1-16 documents on their wiki 
pages) or a comment to the discussion, but the wiki could not tell the design team how long 
each student spent on any one section or item in the wiki. This information could help the 
team understand better which aspects were problematic and which were successful; the 
next implementation of the course will take place in the Canvas platform without university 
support to enable the team to gather such data. 

Task process. Process issues occurred when insufficient instructions were provided or 
students could not or would not try to figure out how to do certain tasks. The team 
considered this problem extensively in the initial design phase and was very explicit about 
providing instructions for each task, assignment, and exercise in the wiki. However, the 
data show that students did not always read the instructions where they were supposed to, 
and the implementation showed us that sound design includes help wherever students say 
they need it, not just where the design team thinks they should have it. For example, the 
wiki organization was thought out carefully, including links to resources, videos, and 
homework. However, the initial schedule, made to show "For Today" and "For Tomorrow" 
in the same line, was confusing to students. Our attempt to make academic connections for 
students by linking today’s learning with the next day’s (Egbert & Ernst-Slavit, 2010) did 
not work well for many students because it was not how they were used to seeing syllabi. 
After the first two iterations, students asked to have a clearer format even though the 
design team felt that clear labels had been used and everything was in a linear table format 
that made sense. In order to be responsive to students’ needs (Meltzer & Hamman, 2004), 
we changed all information to list everything for "Today" in one line, indicating both work 
that should be done prior to class and work that would be done in class, as shown in Figure 
2 above. Students were appreciative not only of the new format but of the responsiveness 
to their needs. 

Content. Most of the issues were around content, and students were unsure where to find 
it and what to do with it. In all, students viewed the wiki 3,025 times, or an average of 55 
times each during the ten-week course session. 
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Questions. Following the flipped literature that says to post questions for group discussion 
led by the teacher, we asked students to post their questions or comments from that week’s 
course material to the wiki for discussion in class. At first we just asked students to post 
about the current unit. We wanted to know their issues with the main topic, but they 
posted about administrative issues, assignment questions, and strategy demonstrations. To 
help students focus on the unit topic, we then gave students instructions to post about what 
they did not understand after trying outside of class to meet the posted learning objectives 
for the unit. We wanted them to be more focused on the main issues so that less time would 
have to be spent weeding through questions about topics that were addressed in other 
places. Students then posted a question for everyone to discuss, but there were still too 
many questions to answer in a discussion, which were often similar to each other. To try to 
solve this issue, the instructor summed up the most common questions and presented the 
resulting question summaries to students in different group configurations: full class 
discussion, in smaller groups, or in pairs and then groups. This system helped the question 
process to be more student-centered because students had more opportunities to 
contribute in small groups and they could also use resources of their choice either on or off 
of their computers to answer the questions. 

According to the course midterm and final evaluations after the first semester iteration, a 
number of the students felt that posting questions was busy work because sometimes they 
did not have a question but were required to post one. In the second semester iteration of 
the course this task was redesigned, for example, by asking students to bring questions to 
class and starting class with small group discussions in which students could control the 
contents. We found that adding more time for course discussion in the flipped classroom 
engaged students in the content more effectively. 

Wiki contents. The wiki served as the main organizational feature of the course and the 
forum for all course content, including handouts used during in-class tasks, student 
questions, and anything other materials that could be posted in digital form. We had 
originally posted as many resources as possible and explained in class what they were and 
how to find them. However, students who used multiple wikis for the other five classes 
they were taking often forgot where to find things or got lost in the wiki. To help students 
find what they needed, we posted the links or materials for each task in multiple places. For 
example, in addition to a separate section for links to the Chinese language Moodle and 
related documents, we made links to the Moodle everywhere it was mentioned on the 
syllabus and schedule. In addition, we added handouts reminding students of tasks we 
taught in class, such as how to make a new wiki page and how to post their fieldwork. We 
found that because not everyone listens to everything in class (and sometimes we did not 
have time to explain something during class time), some students missed important ideas. 
Therefore, we included new information on the front page of the wiki in red in an 
Announcements section so students would know where to go to find what they missed. 
Students noted that they appreciated us making the wiki easier to navigate and more 
transparent to them. One student said she "loved the wiki" because it was so organized, 
unlike in her other classes. 

Instructional goals. The flipped literature suggests that instructors post the course goals 
in conjunction with other posted information so that students know what to do. At first we 
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posted general goals to the wiki for each unit (e.g., Chapter 4: Learn about learning 
objectives ). The students in this class are typically very pragmatically oriented because it is 
their last semester of coursework and they are anxious to graduate; they are interested in 
knowing exactly what to do and how to do it. When students commented that the goals 
were too general, we changed the wiki to include specific objectives for each week, the last 
of which was "create...,” which addressed the activity we were to do in class. However, in a 
teacher education class where there is often not one right way to do anything, the 
instructions and goals on the wiki and the information in what the design team saw as 
quite specific videos were apparently still too abstract for students without more in-class 
explanation. In fact, in one of the sections with 23 students, 20 of them commented on the 
mid-term class evaluation that they needed more instruction, more specific directions, and 
more lecture. We attributed this in part to their cultural backgrounds and experience with 
learning through lectures; the department chair he agreed that the students were 
conditioned to learn from lecture and not used to new ideas or having to find out 
information for themselves. We saw this difference as problematic, particularly in a teacher 
education program, and this made us more determined to support students in doing 
something they saw as "new." We tried to accommodate students within the flipped model 
in weeks five through ten by providing an in-class overview of the next week’s objectives 
and examples of how they could be met. At the end of the course, 20 of the total 
participating students still commented that they needed more lecture. 

Resources. In keeping with the flipped literature that indicates that students should be 
able to access whatever they need for their learning, we posted lists of Web-based and 
print resources for each course unit. At first we posted as many resources as we could find 
in a single list attached to the schedule page, including the videos we had made. However, 
along with the too-general goals, students did not know what to read or watch from these 
long lists or where the vital information was. In response to their confusion, we added the 
video under the objective that it addressed and cut the resource lists down to the most 
essential information. By Week 5 of the course, students felt comfortable with this system 
and were able to navigate the resources more successfully. In final course evaluations, 
several students mentioned in the open-ended comments section that "we were just not 
used to it in the beginning" and as the course "went on it became much better.” 

Reflection to Produce Design Principles 

According to the data, some of the ideas from the flipped literature made a difference to 
student participants’ perceptions of the course. First, the fieldwork was appreciated as an 
"essential" part of their learning to work in diverse communities. Although some of the 
students did not reflect as deeply as we had hoped on their field experiences, course 
documents showed that they appeared to learn something new about themselves and their 
community. These experiences could only happen because of the redesign of the course. 
Second, the logs show that roughly 87% of the participants watched the optional videos 
and provided an answer to a written question that they were useful in understanding the 
course concepts. Although video use was lower for those videos addressing concepts that 
were not directly graded and video use declined over the course of the semester, the 
instructors felt that participants understood the ideas far more quickly than in previous 
semesters. Finally, the addition of the Chinese Moodle helped students clearly and 
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concisely express their ideas about what it felt like to be a language learner and how this 
experience would help them not only to remember how it felt, but also to apply the 
concepts learned in class to help ameliorate some of the issues. In these ways, the flip was 
successful in reaching the goals of the redesign. However, there were problems that we will 
address in the next implementation of the course based on the principles below. 

Address student backgrounds. Although we had general information about the student 
population that the course was to address and the state- and program-defined goals for the 
course, we did not have enough background information at the start of the course to design 
based on the characteristics of these specific participants. Providing choices thus was an 
important principle that we only partly met. Furthermore, we certainly learned during this 
study that the arrangement of information cannot always follow other suggested designs 
profitably; what works in one disciplinary area, particularly one that has a majority of 
"right" and "wrong” answers, will not necessarily work in a more complex disciplinary 
environment such as teacher education. Design may take into consideration the general 
overviews provided in the literature, but specific disciplinary and student needs must also 
be considered. 

A variety of video types, additional in-class activities, and some of the other suggestions 
addressed below will help the course better meet the needs of individual participants. For 
example, we understand that there is no one-size-fits-all video type any more than there is 
one best method of teaching. We will take these ideas into consideration as we proceed 
with the next iteration of course design. 

Consider discipline-specific needs. The actual design of this course was more 
complicated than we anticipated, in part because as pre-service mainstream teachers 
studying ESL rather than ESL teachers, the teacher education students were working on 
creating lesson plans in different disciplinary areas as part of their required work for this 
course. They sometimes could not figure out how the general information about ESL lesson 
components could be applied in their focus area. For example, students developing math 
lessons that were effective for ELLs had more trouble creating language objectives than 
those creating language arts lessons, particularly if the examples in the videos or in other 
resources were in one area rather than another. We need to add examples but do not want 
students to copy them exactly or be limited in their creativity; this is an issue that we are 
still working on and need to address both in-class and in the videos. 

Build in flexibility. Another principle for this teacher education context is that design has 
to allow for changes made at the students’ request. A responsive classroom should be a 
model in teacher education in any case, but using a technology that allows easy change is 
essential in supporting student needs. In our case, wiki use allowed us to include fewer and 
more organized resources, present clear objectives rather than general goals, provide 
different examples and videos as students requested them, and interact with students as 
needed. Most importantly, wiki use allowed us to provide students with a variety of choices 
in materials, timing, and pacing, and such choices are part of an engaging classroom 
environment (Meltzer & Hamman, 2004). 

Provide opportunities for instructional variety. Student course evaluations showed that 
students actually wanted more direct instruction, as they felt that their peers did not have 
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enough experience to help them and they wanted to hear what the "teacher" wanted. This 
may reflect that fact that students were not used to working, or did not want to work, as 
independently as we expected and that the change was uncomfortable for them in some 
ways. This interpretation was verified in discussions with the department chair, leading us 
to assert that any redesign has to either be based in what students know or have done or 
more scaffolded with instruction and practice. The change to providing more specific 
information in class both at the end of the previous lesson and beginning of the current one 
about what to do and how products should look was well-received and did not change the 
idea of flipping. McLaughlin et al. (2014) found this same idea of presenting small bits of 
information, which they called "micro-lectures,” to be very helpful for their students. 

Determine an effective tracking method. Another important principle is that the design 
must include ways to evaluate important aspects. Because we were attempting to follow 
the literature on flipped instruction, we needed to know whether the videos we made were 
supporting learning. However, problems with the Screencast site performance made us 
unable to keep track of who actually completed the videos and who did not. We were able 
to uncover how many views each video had, but no record exists of which videos were 
watched by whom for how long. This important data could help inform not only the success 
of the flip, but also how we might redesign the videos for maximum effect. 
Finally, from our experiences and the data, we can suggest principles that are not 
necessarily new to course design but in this context were crucial: 

• Use a range of technologies that provides the greatest and most reliable access- 
suggest the use of Chrome and Firefox or Safari so students can access the site with 
whatever technology they have, and include a technology FAQ as problems roll in; 

• Provide access to complete instructions (simple for some, more thorough for others) 
in different modes; 

• If students are expected to spend more time outside of class, then less class time is 
necessary (of course this would be more difficult to implement in some settings); 

• Vary video modes - some students wanted to see faces, others like the narrated 
PowerPoint; 

• Even the best organization might need things posted more than once. Wherever a 
handout or site or idea is mentioned, it should have an access point. 

Conclusion 

We had originally tried to follow the ideas in the literature as closely as possible to design 
our flip; because there are no exact steps or design ideas, much of our ideas came from our 
own interpretations of the literature. However, flipped instruction as outlined in much of 
the literature -where students simply watch video lectures outside of class and complete 
activities in class- did not fit the students or context of the ELL teacher education course 
very well. In one part, this could be because teacher education is a complex, open-ended 
field in which there are no exact answers, formulas, or strategies, and static lectures which 
many flipped experiences center on. Teacher education is a process-based rather than fact- 
based field, and the nature of the field makes a difference in how much and to what extent 
videos can teach and what experiences students need, demonstrated in this study by the 
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number of students using both the videos and textbook for content learning. In another 
part this may be because students were already used to the types of in-class activities and 
considered some of the videos to be boring and ineffective, especially when they did not 
work correctly. 

The success of the flip may also depend on where students are in their program, their 
previous experiences, and how flipped instruction is described and supported. In other 
words, in order for flipped instruction to be effective, it must be considerate of context 
rather than applied as a set entity; this is evidenced by the number of changes the research 
team made to the course throughout both semesters. Future studies will include the use of 
student outcome data, formal measures of student engagement, and interviews and/or 
focus groups to obtain an even deeper understanding not only of how students perceive the 
instruction and what principle guide/ should guide it, but also of the impact of this process 
on teacher education student achievement. 

As we conduct additional iterations of the course informed by this and other data, we may 
become more confident that flipped instruction can work well for our teacher education 
courses if attention is paid to the principles mentioned above and if faculty can commit the 
time and effort it takes to observe closely. Because we had a large team working on a single 
class, we could divide up the design tasks in ways that did not overburden any one 
member; teachers who work alone may do well to have a clear plan for design in place 
before starting in order to ease the time burden. Finally, as McLaughlin et al. (2014) 
comment, "the actual practice of off-loading content and engaging in active learning in the 
classroom is far more important than the specific methods we used” (p. 6). In other words, 
although moving the majority of content instruction outside of the class and adding 
additional resources supported the goals for the class, the design principles that we 
explored can help us to develop and to understand, without the rather ambiguous "flipped” 
label, an even more resource-rich, student-centered approach to teacher education 
classrooms in general. 
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